Intelligence

To retrieve threat intelligence from ThreatStream.

ThreatStream provides multiple methods for retrieving threat intelligence. Use the following criteria to determine which method is appropriate for your needs:

You can specify criteria by which the intelligence should be retrieved, as shown in the examples below.

Request

/api/v2/intelligence/

HTTP Methods: GET

For attributes that can be used for filtering, see the Attributes table below.

Response

Format: JSON

Number of rows returned (by default): 20

Attributes: See the table below

Order of attributes: The attributes are sorted by the modified_ts attribute, in descending order.

Attributes

Using “update_id” to Retrieve Large Intelligence Datasets

For the Intelligence API, when the total number of results is greater than 10,000, Anomali recommends using the update_id parameter to return the full dataset through iterative API calls. Using the update_id method ensures large datasets are retrieved without impacting performance.

This method involves appending the following to your API call:

update_id__gt=0&order_by=update_id

After making the first request, locate the update_id of the last returned object (the 1,000th result).
For example:

Use this value for the update_id__gt variable in your next API call. Repeat this process iteratively until no further results are returned.

Example (basic search query):

Run the following API call to retrieve all active intelligence that has been added or modified since January 10, 2018:

curl 'https://api.threatstream.com/api/v2/intelligence/?modified_ts__gt=2018-01-10&status=active&update_id__gt=0&order_by=update_id' -H 'Authorization: apikey <username>:<api_key>'

The update_id value of the last retrieved result—the 1,000th result in this case—is 11849.

{...
  "asn": "3786",
  "classification": "public",
  "confidence": "90",
  "severity": "very-high",
  "source": "<source_name>",
  "source_feed_id": null,
  "srcip": "211.169.202.2",
  "srcipv6": null,
  "state": "active",
  "trusted_circle_ids": null,
  "update_id": "11849"
...
        }

Therefore, the next API call you would make is:

curl 'https://api.threatstream.com/api/v2/intelligence/?modified_ts__gt=2018-01-10T00:00:00&status=active&update_id__gt=11849&order_by=update_id' -H 'Authorization: apikey <username>:<api_key>'

Example (advanced search query): 

Run the following API request to retrieve intelligence of APT IP, Bot IP, and C2 IP types with a confidence score of 90 or higher.

curl 'https://api.threatstream.com/api/v2/intelligence/?q=(confidence>=90+AND+(itype="apt_ip"+OR+itype="bot_ip"+OR+itype="c2_ip")+AND+update_id>0)&order_by=update_id' -H 'Authorization: apikey <username>:<api_key>'

The update_id value of the last retrieved result—the 1,000th result in this case—is 2237854.

{...
  "modified_ts": "2015-04-13T23:00:00.123Z",
  "update_id": 2237854,
  "country": "IN",
  "latitude": 28.6667,
  "type": "ip"
  ...
}

Therefore, the next API call you would make is:

curl 'https://api.threatstream.com/api/v2/intelligence/?q=(confidence>=90+AND+(itype="apt_ip"+OR+itype="bot_ip"+OR+itype="c2_ip")+AND+update_id>2237854)&order_by=update_id' -H 'Authorization: apikey <username>:<api_key>'

Using the Intelligence API in Continuous Operations

If you call the Intelligence API through an automated script for continuous synchronization, follow the guidance in this section to ensure you retrieve all available data within the time frame of interest. Data ingested by ThreatStream is not available instantaneously. To reduce the chance of missing data still being processed by ThreatStream at the time of script execution, use update_id to iteratively retrieve data and set modified_ts__lt to one minute before the current UTC time stamp.

Example:

curl 'https://api.threatstream.com/api/v2/intelligence/?modified_ts__lt=<minus_1_minute>&status=active&update_id__gt=0&order_by=update_id' -H 'Authorization: apikey <username>:<api_key>'

where <minus_1_minute> is the time stamp of the current time minus one minute in YYYYMMDDThhmmss format

Advanced Search Queries

Most filtering criteria can be specified using filter operators described in this document. However, if you need to specify a complex filter, you can use an advanced search query. This method allows you to specify queries similar to those you formulate on the Advanced search screen on the ThreatStream user interface. To specify a filter language query, use this syntax in the API call: &q=<filter_language_query>. The filter operators used for the filter language query are the symbolic form (=, <, >, and so on) and not the descriptive form (exact, lt, gt, and so on) and the query must be URL encoded. See To retrieve intelligence using an advanced search query: for an example.

Examples

As Intelligence API calls can yield large numbers of results, the following examples use the limit attribute to limit the number of results displayed in an API call.

By default, if the limit parameter is not specified, 20 records are fetched at a time. However, when limit=0, a maximum of 1000 the total records that meet the query are fetched.

If the total records returned exceed the defined limit or number greater than 1,000 but fewer than 10,000, use the next field of the returned metadata to make iterative API calls until all records are returned. If total records number greater than 10,000, follow the directions under Using “update_id” to Retrieve Large Intelligence Datasets to fetch complete results.

1. To retrieve the 100 most recent threat intelligence entries in ThreatStream:

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?limit=100' -H 'Authorization: apikey <username>:<api_key>'

2. To retrieve the first 20 threat intelligence entries with a confidence greater than 80, and a severity of “low”:

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?meta.severity=low&confidence__gt=80' -H 'Authorization: apikey <username>:<api_key>'

3. To retrieve all of the intelligence for indicator type "mal_ip" in ThreatStream since a particular date and time:

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?status=active&modified_ts__gte=2014-10-02T20:44:35&limit=0&is_public=1&itype=mal_ip' -H 'Authorization: apikey <username>:<api_key>'

4. To retrieve all of the intelligence in ThreatStream since a particular date AND country = "CN":

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?status=active&date_last__gte=2014-10-02T20:44:35&is_public=1&limit=0&country__exact=CN' -H 'Authorization: apikey <username>:<api_key>'

5. To retrieve all of the intelligence in ThreatStream that is tagged with the value "energetic-bear":

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?status=active&is_public=1&limit=0&tags.name=energetic-bear&' -H 'Authorization: apikey <username>:<api_key>'

6. To retrieve all intelligence that contains "myexample.com":

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?value__contains=myexample.com&limit=0' -H 'Authorization: apikey <username>:<api_key>'

   Tip: For best results, values specified after the contains operator should be at least 3 characters.

7. To retrieve intelligence using an advanced search query:

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?limit=0&q=(expiration_ts>=2022-01-30T08:00:00+AND+confidence>=90+AND+(itype="apt_ip"+OR+itype="bot_ip"+OR+itype="c2_ip"))' -H 'Authorization: apikey <username>:<api_key>'

   For more information on specifying advanced search queries, see Advanced Search Queries.

8. (Only on ThreatStream OnPrem) To retrieve 100 most recent threat intelligence entries stored remotely (on ThreatStream Cloud):

   Copy

   curl 'https://api.threatstream.com/api/v2/intelligence/?limit=100&remote_api=true' -H 'Authorization: apikey <username>:<api_key>'

9. (Only on ThreatStream OnPrem) To retrieve remote intelligence based on a saved search filter whose ID is 398 and which is stored remotely (on ThreatStream Cloud):

   Copy

   curl 'https://<OnPrem_IP_or_FQDN>/api/v2/intelligence/?search_filter=398&remote_api=true' -H 'Authorization: apikey <username>:<api_key>'

10. (Only on ThreatStream OnPrem) To retrieve local intelligence based on a saved search filter whose ID is 1002538 and which is stored locally (on ThreatStream OnPrem):

    Copy

    curl 'https://api.threatstream.com/api/v2/intelligence/?search_filter=1002538' -H 'Authorization: apikey <username>:<api_key>'

Search Filter

To retrieve information on search filters saved from the ThreatStream user interface. After identifying the ID of a search filter, you can execute a query by specifying the search filter ID in the search_filter parameter of an Intelligence API request.

Request

/api/v1/search_filter/

HTTP Method: GET

Response

Format: JSON

Number of rows returned (by default): 20

The following is an example response for an individual saved search filter:

Example

To retrieve a list of search filters saved by your organization:

curl 'https://api.threatstream.com/api/v1/search_filter/' -H 'Authorization: apikey <username>:<api_key>'

Observable Cloning

To create a separate instance of an observable. After identifying the ID of an observable, you can execute a clone request by specifying the observable ID in an Intelligence API request.

When you clone an observable, an import session is initiated, which must be approved to add the cloned observable to your threat intelligence.

Request

/api/v2/intelligence/<observable_id>/clone/

HTTP Method: POST

Response

HTTP 202 Accepted {"job_id": "<background_job_id>", "import_session_id": "<import_session_id>"}

where

job_id is an identifier for the job on ThreatStream.

import_session_id is the ID for the initiated import job. You will need this ID to review and approve the import job.

Example Response:

{"job_id": "368ab6dc-7955-4fd7-aef9-b8cd5496fcb1", "import_session_id": "346928"}

Example

To clone an observable with ID 11202475 which belongs to another organization to your own organization:

curl -X POST 'https://api.threatstream.com/v2/intelligence/11202475/clone/' -H 'Authorization: apikey <username>:<api_key>'

After this, proceed with approving the initiated import job. See Import (with approval) for more details.